Extending resolution within a single imaging frame

The resolution of fluorescence microscopy images is limited by the physical properties of light. In the last decade, numerous super-resolution microscopy (SRM) approaches have been proposed to deal with such hindrance. Here we present Mean-Shift Super Resolution (MSSR), a new SRM algorithm based on the Mean Shift theory, which extends spatial resolution of single fluorescence images beyond the diffraction limit of light. MSSR works on low and high fluorophore densities, is not limited by the architecture of the optical setup and is applicable to single images as well as temporal series. The theoretical limit of spatial resolution, based on optimized real-world imaging conditions and analysis of temporal image stacks, has been measured to be 40 nm. Furthermore, MSSR has denoising capabilities that outperform other SRM approaches. Along with its wide accessibility, MSSR is a powerful, flexible, and generic tool for multidimensional and live cell imaging applications.


Letter to the reviewers
We thank both reviewers for their insightful feedback and efforts which have undoubtedly improved the quality of this work. The thoughtful discussions raised by them have increased the impact of our contributions to the field of optical microscopy. We have addressed crucial aspects related to the comprehensiveness of the manuscript. We appreciate the reviewers for raising critical concerns, which have now been rigorously reasoned through and addressed in the revised versions of the main manuscript, methods section, supplementary material and user's guide for MSSR.
Much care has been taken to address the general concern of reviewer #1 about the claim of whether MSSR can be considered a super-resolution microscopy approach. This led us to revisit our own understanding about the concept of super-resolution, and the ability of MSSR to surpass the diffraction limit. The applicability of MSSR to other, previously unconsidered, microscopy setups and imaging modes was a relevant aspect also brought up by reviewer #1. We have now incorporated four new figures in the main manuscript which show, quantitatively, the spatial resolution enhancement provided by MSSR when operating in combination with confocal, Airy scan, or STED microscopy configurations.
We deeply acknowledge the pivotal suggestions provided by reviewer #2 about the very fundamental mathematical principles over which MSSR operates. The suggestion to associate the PSF to the bandwidth led us to redefine our conceptualization of the h s parameter, which directly improved the results provided by MSSR. Moreover, the questioning about the kernels motivated us to seek more convenient alternatives for explaining the basis of our algorithm. We appreciate reviewer #2 for pointing out errors in some of the formulas shown in the supplementary material, which have now been corrected and will help the readers understand the principles of MSSR more clearly.

Reviewer comments Reviewer #1 (Remarks to the Author):
This work adapts the concept of mean shift for designing an elegant and sophisticated algorithm for processing microscopy images. The authors not only explain the development of the concept to the proposed algorithm, that they call MSSR, but also provide pertinent implementation details, open software, operation manual, and a variety of conceptual and experimental examples.
My main concern about the work is the following. The single-frame MSSR algorithm performs smoothing, sharpening of features and noise suppression simultaneously, but not super-resolution. In this sense, it is similar to radiality transform used in SRRF, where super-resolution is derived from temporal analysis and radiality transform mainly achieves smoothing and sharpening. It may also be considered as a sophisticated and numerically stable approach for performing deconvolution. The fact that the algorithm works by the sharpening of features and not inherently performs super-resolution is also manifested in the fact that single-frame MSSR obtains only 1.6 times the resolution enhancement. This is related to the threshold of 65% in line 227 and clearly evident at the Rayleigh resolution limit. Any further superior resolution is achieved only through temporal analysis, or by applying MSSR on images obtained by other super-resolution algorithms. If the authors disagree with me, I request the authors to show that sf-MSSR (of any order of authors' choice) can resolve two fluorophores at distance, say 0.8 times the resolution limit. Otherwise, it is important that the paper be modified to remove the super-resolution claim or to restrict the claim of super-resolution to only t-MSSR.
Beyond that there are several minor concerns, which I believe are easily addressable: 1. It will be useful to include the resolution limits for the examples of nanorods and crispr examples, which are specifically being used to show super-resolution. For example, in Figure 2, it would have been nice to see also the resolution limit in the caption in order to appreciate the order of enhancement.
2. No details of the parameters used for the other algorithms are provided. The details of simulated examples are unavailable or not straightforward to find.
3. Lines 73-74, it might be useful to clearly restrict the super-resolution algorithms on which MSSR can be applied. My impression from the theoretical development is that MSSR cannot be applied to super-resolved imaged generated by localization microscopy and may be applicable to STED only if very small scanning step size is used. It may also have limited or no applicability to confocal and Airy scan microscopes (even though they may not be considered super-resolution microscopes). 4. It will be appreciable if the authors can comment on the applicability or potential of adaptation for 3D microscopy, including 3D epifluorescence stacks, light sheet microscopy generated stacks. 5. It seems like the sentence in the lines 103-105 (The MS is a vector ..) is in a direct conflict with the phrase in the lines 106-107 (thus is not necessarily ...). It will be useful to either elaborate and clear the confusion or remove the second phrase completely. 6. There are quite a few places where notations appear before they are introduced. For example, MSSR 0 or sf-MSSR. There are some incorrect cross-references, for example, Supplementary Figure S19-20 in lines 257-258. In line 157 of the supplementary file, the authors use 'Sections' instead of 'Supplementary notes'. 7. The analysis discussed in lines 203-208 is not easy to follow. It will be useful to rewrite this paragraph for better readability and apprehension. 8. Gattaquant and CRISPR samples, simulation samples, other control samples: It might be useful to either include the temporal fluctuation characteristics plot or discuss the nature of fluctuations (photokinetics) of the fluorophores. Are they blinking, what are the dark periods, etc? That definitely has an implication on how many frames and what resolution to expect. These factors, including the density of fluorophores emitting per frame may have a significant consequence on, for example, Figure S21. 9. Figure S21 and Figure S29 need better presentation. It is difficult to say one plot from the other. In Figure S21, if need be, authors may want to limit the number of frames to 25, but better visibility of each plot is quite important. Potentially a smaller but more important subset of SNR values may be used. 10. I suppose including one result on biological samples in the main manuscript may be useful in improving the visibility of the manuscript when published.
In all, I appreciate the work presented in this article. However, the authors need to address my major concern on super-resolution claim. While super-resolution is attractive, this work holds value even without super-resolution claim on sf-MSSR and authors have done a good job in demonstrating the value. So, I believe that this manuscript is a significant contribution even without super-resolution claim and strongly encourage the authors to be technically correct on this matter. If this concern is addressed, and possible improvements on the minor comments are incorporated, I will be happy to recommend this manuscript for publication.

Reviewer #2 (Remarks to the Author):
This work presents MSSR, a principle and an algorithm for sharpening a digital image generated from diffraction of point sources. Given that the diffraction can be modelled with a Gaussian spread (PSF G ), the pixel intensities I(p) is a superimposition of spreads from emitters. The goal is to identify the positions of the emitters, or at least reduce the spread. MS stands for mean shift, or sometimes called the "blurring process". The idea is that by blurring, one can sharpen an image. The phase "sharpening by blur" is a procedure in Photoshop, at least since 2017.
The idea is that by selecting a "spatial bandwidth", h s , (and on top of that, a "range bandwidth", h r ), one can identify a set of intensities in the color space, compute their mean, and then perform a mean shift, or update each pixel's intensity to the mean. If the pixel is not too far from the position of the emitter, the mean shift with the same Gaussian kernel may be in the same direction of the gradient in the intensity space, and the intensity may increase. Otherwise, the mean may go to the opposite direction and the pixel may lose its intensity. It would be nice and also important to relate the transformation of the point spread function (PSF G ), essentially the sigma parameter, to the mean-shift parameter h_r (or maybe h s ) in this work. This should be easy but is not done here.
Supplementary note 3 presents the "meanshift imaging model". Each pixel is a pair [position, intensity] of (spatial, range) components. Given a pixel [x, y], a neighborhood is defined in (S17), which includes pixels within h s spatial distance and also with h r range distance. The asterisk in B* probably excludes the host pixel. (S17) also excludes all neighboring pixels with the same intensity, which could be an error in the formulation.
(S19) defines the kernel density estimate at each pixel position. It may contain a typo: By V do you mean B?
Since the positions of pixels are the grid, the k_s part of (S18), and the g s part of (S23) are constants and the two-component profiles k = k s .k r and g = k s '.k r ' are unnecessary and indeed when you reach (S24), only intensity is left. q n in (S25-27) will be the same with or without k_s or k s '. In other words, the only component in [x, y] that moves is y, and no mean shift is done on x.
Since x is not shifted, and thus all dx are zero, Supplementary note 4 and its theorem, which relies on the shift of x, becomes irrelevant.

Response to Reviewer #1
Remarks to the Author: This work adapts the concept of mean shift for designing an elegant and sophisticated algorithm for processing microscopy images. The authors not only explain the development of the concept to the proposed algorithm, that they call MSSR, but also provide pertinent implementation details, open software, operation manual, and a variety of conceptual and experimental examples. Q1.1. My main concern about the work is the following. The single-frame MSSR algorithm performs smoothing, sharpening of features and noise suppression simultaneously, but not super-resolution.

A1.1.
Defining the concept of resolution in optical microscopy is not a simple task. It has been the subject of scientific scrutiny for a long time. However, there is no consensus in the scientific community about which approaches are considered super-resolution (SR), as discussed in [1]. Some scientists appraise that surpassing either Abbe's or the Rayleigh limits can achieve SR [2,3]. There is a view about the problem of spatial resolution, which makes a further segmentation of the resolution concept into three main categories: extended resolution (extended-R), enhanced resolution (enhanced-R), or SR [4]. Extended-R and enhanced-R, include those optical approaches that extend or enhance optical resolution down below Abbe's or the Rayleigh limits, i.e., by means of using clever illumination or detection schemes and image processing (i.e., 4Pi, I 5 M, SIM, Airyscan), but which never surpass the diffraction barrier [5][6][7][8]. SR is accomplished only by isolating the signal of individual fluorescent emitters transiting between ON/OFF either through deterministic (STED) or stochastic processes (i.e., STORM, PALM, PAINT, FF-SRM) [9][10][11][12][13][14], or by a combination of both (i.e., RESOLFT, MINFLUX and MINSTED) [15][16][17]. In the former definition of optical resolution [11], sf-MSSR n and t-MSSR n , surpass the Rayleigh limit [18]; hence they would be considered SR approaches. However, if we segment the problem in extended-R, enhanced-R, or SR, then sf-MSSR n should be considered either extended-R or enhanced-R as its procedure embraces the analysis of diffraction-limited data. In stark contrast, t-MSSR n is a FF-SRM approach that overcomes the diffraction barrier by statistically analyzing fluorescence intermittency in time.
The problem of spatial resolution in optical microscopy can also be addressed from a statistical point of view. In the case of fluorescence microscopy, the process of photon emission from point sources (fluorescence emitters) can be considered as a discrete distribution of information, where the unitary element of the distribution is the photon [19]. In this scenario, spatial resolution gets reduced to finding modes of information, regardless of the shape of the distribution, hence, disconnecting the problem of optical resolution from the diffraction boundary [20]. Furthermore, by computing the local magnitude of the Mean Shift vector, MSSR creates a probability distribution of fluorescence estimates whose local magnitude peaks at the source of information. As a result of that, the spatial distribution of information becomes 'refined', i.e., for a Gaussian distribution of fluorescence, its width shrinks.
Whether the refinement of information provided in sf-MSSR n should be considered SR will depend on the scientific point of view embraced to assess the problem of spatial resolution in optical microscopy. Horever, we appreciate the feedback provided by reviewer #1, which led us to revisit our conceptualization of the spatial resolution in optical microscopy and to develop a rationale for the meaning of deconvolution at the nanoscopic scales (see A1. In the following lines, we provide extra comparative data against: Wiener deconvolution [21], Richardson-Lucy deconvolution [22,23] and the Radiality Maps [24]. Q1.1.1. In this sense, it is similar to radiality transform used in SRRF, where super-resolution is derived from temporal analysis and radiality transform mainly achieves smoothing and sharpening. A1.1.1. We agree with reviewer #1 because MSSR 0 and the Radiality Maps (RMs) are similar in the sense that both perform sharpening and smoothing. However, we disagree with reviewer #1 that super-resolution, in either MSSR or SRRF, is mainly derived from the temporal analysis. Gustafsson [25]. Moreover, the authors of SRRF indicate that when using a pixel-by-pixel projection, such as the temporal radiality maximum (TRM) or temporal radiality average (TRA), there is no further gain in spatial resolution.
In the same note (Supplementary note 1, specifically in section "Temporal Analysis -Generating the SRRF Map"), the authors of SRRF state that "In datasets where peak separation is greater than 0.7 times the FWHM of the PSF, emitters can be directly resolved by the radiality maps without further enhancement brought by higher-order temporal correlations", hence, either TRA or TRM serve the purpose of noise and artifact suppression, but not to exceed the spatial resolution gain provided by the RMs. Figure A1.1a shows that both MSSR 0 and the RMs overcome the Rayleigh diffraction limit, however, the RMs produce undesired artifacts which are absent when using MSSR 0 ( Figure A1.1b). The spatial artifacts introduced by SRRF have been revisited elsewhere [26]. Nonetheless, and despite the undesired imaging artifacts introduced by computing the RMs, they extend spatial resolution by analyzing a single diffraction limited image.
MSSR 0 reliably provides spatial resolution gains, free of image analysis artifacts, down to the Rayleigh limit, hence, allowing the study of nanoscopic regimes at the boundaries of the Sparrow limit ( Figure A1.1). MSSR of higher orders (MSSR n : n > 0) encompass further analytical operations (subtraction, complement, negative and intensity weighting) which allow the observer to distinguish between single emitters separated by ∼ 0.66 times the Gaussian FWHM of the PSF (1.55 times the PSF s.d. σ). Figure A1  Resolution increase provided by deconvolution methods, radiality maps and sf-MSSR n . a) All methods tested decrease the dip successfully. The horizontal line represents the Rayleigh limit in relation to the dip height (for the sum of Gaussians, 2.5σ). This value decreases the most when using radiality maps and MSSR, where the best result reaches Rayleigh limit = 1.55σ (1.61 times smaller), provided by MSSR of third order (orange line). b) Bidimensional representations of the reconstructed images provided by each method tested, for emitters separated at distances of 1.55σ, 1.9σ and 2σ. Simulated emitters were generated with the gaussian distribution with a sigma of 121.10 nm (~10 pixels) using the following parameters: Refractive index of medium = 1.33, refractive index of oil = 1.515, Numerical aperture = 1.4, wavelength = 610 nm, pixel size 11.7 nm. Q1.1.2 It may also be considered as a sophisticated and numerically stable approach for performing deconvolution. Figure A1.1 shows that none of the tested deconvolution algorithms, except for single-frame MSSR, reliably overcomes both Sparrow's and Rayleigh's limits of optical microscopy [18,27]. Hence, this demonstrates that MSSR generates results that reach SR scales (see A1.1.1 for further discussion about the meaning of the concept of SR).

A1.1.2.
We agree with Reviewer #1 that MSSR aims to revert the effect of diffraction on optical microscopy, so it must be considered a deconvolution process. However, what makes sf-MSSR n unique is that it restores spatial information below the Sparrow limit. For example, Figure A1.1 shows that Wiener deconvolution partially deconvolves the diffraction effect but without a dramatic increase in spatial resolution. Interestingly, Richardson-Lucy deconvolution provides a noticeable increase in resolution at the boundaries of the Rayleigh limit but fails to restore spatial resolution down the Sparrow limit ( Figure  A1.1). With such data in hand, we conclude that sf-MSSR n is the first deconvolution approach of optical microscopy that restores spatial information and can achieve nanoscale information.

Q1.1.3 The fact that the algorithm works by the sharpening of features and not inherently performs
super-resolution is also manifested in the fact that single-frame MSSR obtains only 1.6 times the resolution enhancement. This is related to the threshold of 65% in line 227 and clearly evident at the Rayleigh resolution limit.

A1.1.3.
We acknowledge reviewer #1 for stressing this point, which has also been raised by other readers of the preprint posted at bioRxiv [28]. We wish to clarify that the 65 % is not a threshold taken over the initial Gaussian distribution, as MSSR is not performing straightforward thresholding and normalization processes over it. Instead, MSSR shrinks the Dip formed in between the distribution shaped by two neighboring emitters, by computing the negative of the MS (-MS) and introducing a negative constraint over -MS, which is followed of by normalization and multiplication process (see steps i-iv in Supplementary Figure   . Hence, it is tempting to conclude that there is a noticeable increase in resolution as the Rayleigh limit has been surpassed. By repeating the same procedure using a joint distribution shaped by two adjacent emitters located at the Sparrow limit there is no further gain of resolution as the dip remains constant, taking the value of 1 ( Figure A1  Thresholding and normalizing processes are not enough to overcome the Sparrow limit. This figure shows the sequential decrease in dip height between two intensity Gaussian distributions from simulated point emitters (a,e). Thresholding and normalizing (b,f) provides an initial reduction in dip height, which continues to decrease as MSSR of higher order is applied (c,d,g,h). Upper and bottom rows depict the relative distance between the peaks of both distributions when either the Rayleigh (top) or Sparrow (bottom) criteria are met. Worth noticing is that higher-order MSSR provides a complete dip collapse to zero, separating the distributions of both emitters completely.
The sentence "This threshold operation exerts influence on structures at σ, at about 65% of the intensity distribution of the emitters; values below this threshold will be considered as noise and set to zero value" (lines 226-227) ..., has been substituted by the following sentence: "The goal of this step is to remove an artifact caused by the MS calculation itself when applied to Gaussian and Bessel distributions. When calculating the complement of the resulting distribution, a valley (or depression artifact) is generated between the peak intensity and the tails, which lies in the negative values and is referred to as negative constraints. After this step, the artifact is removed".

Q1.1.4. Any further superior resolution is achieved only through temporal analysis, or by applying MSSR on images obtained by other super-resolution algorithms.
A1.1.4. We partially agree with reviewer #1. Using the TRA or TRM pixel-wise temporal functions in combination with MSSR n (t-MSSR n ) does not provide extra resolution enhancement. TRA provide t-MSSR n reconstructions less influenced by noise (recommended in imaging scenarios of marginal signal-to-noise ratios) and TRM delivers reconstructions whose intensity distribution is less dominated by constantly emitting sources, i.e., fiducial markers. In stark contrast, higher spatial resolution regimes, ∼ 0.15 times the FWHM of the PSF (PSF σ: [0.4 -0.5]), are achieved by t-MSSR n when encompassing pixel-wise temporal functions of higher-order statistics such as Variance (Var), Coefficient of Variation (CV), or SOFI (TRAC2-4).

Q1.1.5.
If the authors disagree with me, I request the authors to show that sf-MSSR (of any order of authors' choice) can resolve two fluorophores at distance, say 0.8 times the resolution limit. Otherwise, it is important that the paper be modified to remove the super-resolution claim or to restrict the claim of super-resolution to only t-MSSR. A1.1.5. We find of great value all the inquiries raised by reviewer #1, which strengthened the scientific quality of this research. We hope to convince the reviewer that the answers A1.1.1, A1.1.2, A1.1.3, A1.1.4 and this one, provide enough scientific arguments to justify the super-resolution claims about MSSR.
In A.1.1.1 we develop the idea that if we keep to the operational definition of spatial resolution provided by both the Rayleigh and Sparrow limits, then we can conclude that sf-MSSR n can be considered as a SR approach. We also mention that we are aware and acknowledge that there are other definitions of resolution which consider physical aspects of the wave nature of light. Such arguments lead to the three categories to classify technologies that provide extended, enhanced, or super-resolved images.
We have changed some instances of super-resolution claims used in the main text, and replace them for the concepts of extended-R, enhanced-R and SR: • The term extended resolution (extended-R) is used when describing the gain of spatial resolution provided by sf-MSSR n . • The term enhanced resolution (enhanced-R) is used when implementing sf-MSSR n in combination with other methods which by itself provide extended or enhanced resolution, such as SIM (and its variants), and Airyscan. • The term enhanced resolution (enhanced-R) is also used when implementing sf-MSSR n in combination with other super-resolution methods (i.e., STED). The resulting image is considered as a super-resolved image (SR). • The term SR is also used for any fluorescence fluctuation based super resolution microscopy (FF-SRM) approach, which includes t-MSSR n , SOFI, SRRF, ESI and MUSICAL.
We hope that the use of segmented definitions for resolution will minimize confusion with the readers of the manuscript. Having stated that, we have included extra further experimental or analytical data which strengthen the argument that sf-MSSR n restores nanoscopic information down below the Sparrow limit. Now, we would like to tackle the following inquiry of reviewer #1: … "I request the authors to show that sf-MSSR (of any order of authors' choice) can resolve two fluorophores at distance, say 0.8 times the resolution limit".
To address the claim of resolving two fluorophores at 0.8 times the resolution limit, we will recall the theoretical case at the Rayleigh limit for a GATTA-SIM 140B nanoruler ( Figure 2c of the main manuscript) with λ em = 525 nm and NA = 1.4, is d = (0.61*525)/1.4 = 229 nm (table A1.1.5). Computation of resolution on sf-MSSR 0 using the Rayleigh criterion (at 0.81 of the Dip) gives a spatial resolution of 160 nm, which corresponds to a resolution change to 0.69 times (0.69x) the resolution limit. Using higher orders of sf-MSSR n with 0<n < 4, gives a resolution change of 0.66 times the resolution limit (table A1.1.5).
With the data shown in table A1.1.5 we conclude that the inquiry of reviewer #1, about the resolution change provided by MSSR has been satisfied using simulated data. Table A1.1.5 also shows Resolution (1/K 0 ) computed from the maximum spatial frequency (K 0 ) measured on the GATTA-SIM 140B nanorulers (experimental data) through decorrelation [29] using the Image Decorrelation plug-in for Fiji/ImageJ. Note that sf-MSSR n dramatically reduces the value of 1/K 0 as a function of the n-order.   To further test the attainable resolution by sf-MSSR n we used the ArgoLight test slide, acquiring images of the pattern formed by gradually spaced lines of fluorescent molecules (Argo-SIM, pattern E). The distance between lines increases from 0 to 390 nm with a step change of 30 nm: 0 nm, 30 nm, 60 nm, etc. Confocal Images (Zeiss LSM880) were acquired using a Plan-Apochromat 63x/1.4 Oil immersion objective, exciting the micropattern with the laser 405 nm and detecting the fluorescence in the range 420 -480 nm. The pixel size was 44 nm. Figure A1.2 shows the application of sf-MSSR n to the confocal microscopy images of the Argo-SIM micropattern. As expected, the Confocal acquisition allows to resolve parallel rows of fluorophores located at 240 nm. Remarkably, sf-MSSR 0 processing extended spatial resolution down to 0.5 times the confocal resolution, allowing to discriminate parallel rows of fluorophores located at 120 nm. It is worth mentioning that this improvement comes at zero hardware cost, compared to other methods requiring specific optics/detectors such as the Airyscan [30] and Re-scan confocal microscopy [31]. Higher orders of sf-MSSR n create a saddle point between parallel rows of emitters located in the range of 60 -90 nm at the boundaries of the Rayleigh limit. Figure A1.2 has been added to the manuscript as Figure 3. In conclusion, it was shown that sf-MSSR n can extend the resolution limit in both simulated and real experimental conditions down below 0.8 times the resolution limit of optical microscopy. Figure A1.1.5 and its corresponding description has been incorporated into the main manuscript as -SNR calculation for raw data.

Q1.5. Lines 73-74, it might be useful to clearly restrict the super-resolution algorithms on which MSSR
can be applied.

A1.5.
MSSR addresses the problem of spatial resolution in optical fluorescence microscopy from the statistical point of view, where the process of photon emission from punctual sources is considered as a discrete distribution of information, provided by detected photons. Enhanced or extended resolution is achieved by finding local modes of information without taking any prior about the local shape of the distribution. The latter makes the MSSR principle compatible with any imaging approach where the distribution of information is of discrete nature (i.e., photons), and emanates from discrete sources (i.e., fluorophores). In the discussion section of the manuscript, we discuss some extra imaging applications suitable for MSSR processing, including broader applications of optics, not restricted to optical microscopy.
Q1.6. My impression from the theoretical development is that MSSR cannot be applied to super-resolved images generated by localization microscopy A1.6. Agree. SMLM images are built through rendering localization tables of single emitters. Normally, this processing encompasses either a smoothing or an interpolation step, where the degree of smoothing can be set up according to the precision of the localization of single emitters [32,33]. Further applying MSSR to these images would decrease the FWHM provided by the selected rendering algorithm.
We note that an interesting avenue might be using sf-MSSR n prior to rendering a SMLM reconstruction. The Mean-Shift has been used for SMLM [34][35][36], it has been recently implemented for drift correction [37]. We have not assessed whether sf-MSSR n preprocessing of DL images would improve the resolution of SMLM reconstruction. Nonetheless, we have addressed a similar problem found in the realm of Single Particle Tracking (SPT) of diffraction limited objects. Supplementary Figures S35 and S36 show that MSSR improves SPT by increasing the localization precision and decreasing the FWHM of their individual fluorescence distributions. Based on that, it may be possible to pre-process SMLM data with MSSR and then perform SMLM analysis afterwards aiming to gain further resolution through improving the capabilities of SMLM algorithms to localize individual emitters. We are exploring this further for a subsequent work.

A1.7.
To examine achievable resolution by sf-MSSR in combination with Stimulated-emission depletion (STED) microscopy we set to explore the capabilities to discern posttranslational modifications of histone proteins (H3K27me3, H3K27ac) within the 2-cell stage embryo in mice. Immunofluorescence imaging was carried out using a STEDYCON mounted on an upright Zeiss microscope in confocal, or STED modes. Samples were imaged with a Zeiss 100x 1.46 NA objective, 20 nm pixel size, 20 nm pixel size, 5 us pixel dwell time, 15-line accumulations and a pinhole of 64 um. Immunofluorescence was performed as in [38]. Primary antibodies used were anti-H3K27me3 (Abcam ab6002) and anti-H3K27ac (Active Motif, 39034), both at 1:200 dilution. Secondary antibodies used were anti-mouse labeled with STAR Red and anti-rabbit labeled with STAR Orange. STED laser powers were 3% for the 640 nm and 775 nm 96.5% for the STAR red channel, whereas for the STAR Orange channel was 7.8% of the 561 nm laser and 100% for 775 nm. Figure A1.7 a, b, show that STED, but not confocal imaging allows to discern chromosomal territories within the chromatin of 2-cell stage embryo, as observed by the lack of colocalization of acetylated (ac, i.e., 'active') or methylated (me, i.e., 'inactive') chromatin states, H3K27me3, and H3K27ac respectively. Remarkably, Figure A1.7c shows that sf-MSSR 1 processing of the confocal image allows to reach a similar experimental conclusion, that there is no colocalization between H3K27me3 and H3K27ac signals. Figure A1.7 also shows the result of post-processing of the STED image with sf-MSSR 1 ( Figure  A1.7g). We note an increase in contrast and resolution from confocal to STED and then to STED + sf-MSSR 1 (compare panels e, f, and g in Figure A1.7). To assess the resolution change provided by either STED or STED + sf-MSSR 1 we used the image decorrelation approach [29], which allows to compute the maximal observable frequency in an image (K 0 ) as a proxy of spatial resolution (Resolution 1/ K 0 ). The bottom panels of Figure A1.7 e, f, and g, show the resolution computed at a confocal image (315 -330 nm), STED (80 -100 nm) and STED + sf-MSSR 1 (19 -22 nm). With these results, we conclude that sf-MSSR 1 processing of STED data provides a further increase in spatial resolution. Figure A1.7 and its corresponding description has been incorporated into the main manuscript as Figure 6.  c) and d) show the corresponding super-resolved scenes provided by post-processing of the confocal and STED images with sf-MSSR 1 , respectively. The bottom row shows the sequential increase in resolution when comparing the same sample imaged with a (e) confocal microscope, an (f) STED microscope and (g) after processing the STED image with sf-MSSR 1 . For each image, the resolution range (in nanometers) was assessed using the ImDecorr algorithm in ImageJ. Immunofluorescence Imaging was carried out using a STEDYCON mounted on an upright Zeiss microscope in confocal or STED modes. Samples were imaged with a 20 nm pixel size. Primary antibodies: anti-H3K27me3 (Abcam ab6002) and anti-H3K27ac (Active Motif, 39034). Secondary antibodies: anti-mouse labeled with STAR Red and anti-rabbit labeled with STAR Orange. The parameters used for sf-MSSR processing were AMP = 5, order = 1, FWHM of PSF = 4, interpolation = bicubic, mesh-minimization = true. Q1.8. It may also have limited or no applicability to confocal and Airyscan microscopes (even though they may not be considered super-resolution microscopes). Figures A1.2 and A1.7. show that sf-MSSR n extends the spatial resolution of confocal microscopy images. Figure A1.8 below shows examples of the application of MSSR to Airyscan microscopy data (same sample data as in Figure A1.2). The Argo SIM micropattern E was imaged using an Airyscan detector with the same settings for laser power, detector gain, image format, bit depth, and line averaging of Figure A1.2, using a selective optical filter BP 420-480 + LP 605, and processing the images with the Airyscan algorithm set on strength parameter 6. The pixel size was 44 nm. Figure A1.8, shows the application of sf-MSSR n to the Airyscan microscopy images of the Argo-SIM micropattern. Within the Airyscan processed image it is possible to resolve parallel rows of fluorophores located at 180 nm, but not 120 nm or less. sf-MSSR 0 processing enhanced the spatial resolution of the same Airyscan data, allowing to discriminate parallel rows of fluorophores located at 120 nm, or less ( Figure A1.8 a, b). Higher orders of sf-MSSR n create a saddle point between parallel rows of emitters located in the range of 60 -120 nm ( Figure A1.8 b, c). Remarkably, compared with the confocal original data in Figure A1.2 where the last resolvable line pair is the one corresponding to a distance of 240 nm, the value of 120 nm obtained in Figure A1.8 by applying sf-MSSR 0 to Airyscan processed data corresponds to a 2-fold improvement in resolution. The first reported applications of Airyscan technology allowed an improvement in resolution of 1.7X [30], while only following protocols claim that a 2-fold improvement might be achieved [39], compared to standard confocal detection.

A1.8.
We then applied sf-MSSR 0 on a SIM image of sister meiotic chromatids of mouse chromosomes [40]. Similar to the results obtained above with Airyscan, Figure A1.8d shows that processing with SIM images with sf-MSSR 0 enhances both the contrast and resolution of the final image. Figure A1.8. has been incorporated as Figure 4 for the main manuscript. Q1.9. It will be appreciable if the authors can comment on the applicability or potential of adaptation for 3D microscopy, including 3D epifluorescence stacks, light sheet microscopy generated stacks.

A1.9.
Since MS theory is not limited by the number of dimensions, the extension of MSSR to three-dimensional (3D) can be achieved by adding a parameter h s that also includes the coordinates on the z axis. However, this extension is not trivial. The diffraction in the axial plane is greater than the lateral plane, so the parameter h s for the z-axis needs to be greater compared to the hs for the xy axes.
Hence, the value of h s for the z-axis will be related to the resolution limit of this axis. On the other hand, an important feature to get a reliable reconstruction when image magnification is performed, is that the number of pixels for each dimension must be increased proportionally. Figure A1.9 shows a comparison of 3D reconstruction from a stack of epifluorescence images of bovine pulmonary artery endothelial (BPAE) cells. When a 2D image is processed by sf-MSSR 1 , an improvement in spatial resolution and contrast is observed ( Figure A1.8 a, b). On the contrary, when a stack is used for a 3D representation (images taken at z-planes), the resolution obtained by sf-MSSR along the z-axis is not as refined as in the xy-plane ( fig. A1.9 c, d). Nonetheless, a noticeable increase in contrast is attained in the overall data set ( Figure A1.9 d, see supplementary videos 11 and 12). To further extend the resolution of DL images in 3D, it is necessary to extend the current implementations of the MSSR algorithm to account 3D information for MSSR processing. Therefore, it is necessary to make a detailed study for MSSR in 3D. Figure A1.9. has been incorporated into the Supplementary Material as Supplementary Figure S41.  3D stack (a,c) and the super-resolved (sf-MSSR) counterpart (b,d) is provided. BPAE cells were labeled with MitoTracker™ Red CMXRos (mitochondria, red), Alexa Fluor™ 488 -Phalloidin (actin, green) and DAPI (nuclei, blue) and imaged in a ONI Nanoimager using epifluorescence mode. 20 steps in Z were acquired to achieve a 1 microns 3D section of the sample. Bottom row provides a rotated view of the sample. sf-MSSR parameters: AMP = 10, FWHM of PSF = 2.77, order = 1. 3D reconstruction was obtained using napari viewer and adjusting the voxel size through the console commands. Scale bar = 10 μm.
The outcome of sf-MSSR processing (2D) of 3D confocal or 3D light sheet microscopy is discussed in the Supplementary Note 11 (Supplementary Figure S40, Supplementary Videos 11 and 12). As in the epifluorescence case, the axial resolution obtained by sf-MSSR 1 is not as refined as in the lateral plane, even so, a characteristic increase in contrast is always attained in the overall data set.
The MSSR principle can be developed for explicit volumetric imaging by means of using an asymmetric kernel, which can be defined following the 3D lateral-axial aspect ratio of the PSF, but also, encompassing possible deformations of axial symmetry of the PSF (due to optical aberrations introduced by the sample or the imaging system). Handling optical aberration in 3D light sheet microscopy is an active field of research (i.e., deconvolution or adaptive optics), where we want to contribute through the further development of MSSR applications in 3D.

Q1.10. It seems like the sentence in the lines 103-105 (The MS is a vector...) is in a direct conflict with the phrase in the lines 106-107 (thus is not necessarily...). It will be useful to either elaborate and clear the confusion or remove the second phrase completely.
A1.10. We agree with reviewer #1. The text in the lines 103-105 is confusing and introduces a conflict with other sections. For this reason, these lines were rewritten. The older text was: "The MS is a vector that always points towards the direction of the intensity gradient and its length provides a local measure of the fluorescence density and brightness [17][18][19]. Since the MS lies within the gradient space, its values depend on the difference between the central pixel of the neighborhood and the surrounding pixels and thus is not necessarily linked to the fluorescence intensity values of the raw image." The corrected text is: "The MS is a vector that always points towards the direction of the intensity gradient and its length provides a local measure of the fluorescence density and brightness [17][18][19]; its magnitude depends on the value difference between the central pixel of the neighborhood and the surrounding pixels." A1.11. We thank the reviewer for calling our attention to check for such kinds of errors. All notation and reference-related errors have been revisited and corrected. Some of them are: (a) MSSR of zero order term is now properly introduced and correctly denoted as MSSR 0 (b) Single-frame MSSR analysis of a given order n term is now properly introduced and correctly denoted as sf-MSSR n (c) Temporal MSSR analysis of a given order n term is now properly introduced and correctly denoted as t-MSSR n (d) Otherwise, using MSSR without prefixes or superscripts makes reference to the super resolution algorithm. (e) References to figures in the supplementary material are now correctly written in the main manuscript or supplementary material. Every supplementary "Figure X" has been rewritten as " Supplementary Figure X". (f) In the Supplementary Material, every "Section X" has been rewritten as "Supplementary Note X".
Q1.12. The analysis discussed in lines 203-208 is not easy to follow. It will be useful to rewrite this paragraph for better readability and apprehension.
A1.12. The paragraph has been rewritten to provide a clearer explanation of the results shown in Figure  6 (previously Figure 4). On the other hand, because the algorithm parameters changed, the distance values among nanorulers have also been updated. The older text was: "To determine the distance between two dCas12a sites along the DNA chain we obtained the distribution of distances between dCas12a binding sites taking in consideration their unidimensional association to a semi-flexible polymer such as the DNA [28]. Estimated distances after t-MSSR 3 -Var in the CRISPR/dCas12a nanoruler are 85 ± 14 nm, 152 ± 21 nm, 232 ± 37 nm (Figure 4d). These results confirm that t-MSSR 3 can successfully resolve nanoscopic distances." The corrected text is: "Once the CRISPR/dCas12a nanorulers were super-resolved by t-MSSR 3 -Var (Figure 7c), we scanned all the individual emitters in close proximity to determine the average distances between different dCas12a associated to the dsDNA binding sites (theoretically interspaced by 100 nm, Supplementary Figure S41). Since DNA in solution is a semi-flexible polymer [4], the measured distances resulted in three different distributions: 91 ± 31 nm, 220 ± 52 nm, 323 ± 19 nm (Figure 7d), which accounts for dCas12a separated by one, two and three binding sites along the dsDNA, respectively. These results confirm that t-MSSR 3 can successfully resolve nanoscopic distances." A1.13. We acknowledge reviewer #1 for raising this comment. Analyzing the temporal dynamics of fluorescence is central to achieving the highest attainable gain of resolution by fluorescence fluctuation based super resolution microscopy (FF-SRM) approaches (reviewed in [37]). FF-SRM methods analyze higher order temporal statistics looking to discern correlated temporal information (due to fluorescent fluctuations) from uncorrelated noise (i.e., noise detector). t-MSSR n encompasses a pixel-wise temporal analysis which delivers further resolution gain, hence, t-MSSR n can be considered an FF-SRM approach.  Figure A1.13 shows the pixel-wise temporal dynamics of two GATTA paint ranorulers imaged for 15 seconds at a 50 ms interval. In the provided examples (nanorulers 1 and 2), the temporal dynamics of a nanoruler containing three binding sites can be studied by analyzing the signal fluctuations at two pixels (px1 and px2). Note that, by analyzing either px1 or px2, it is difficult to distinguish the temporal dynamics of individual PAINT binding sites. Notably, sf-MSSR 3 processing allows to untangle, in space and in time, the fluorescence dynamics at individual PAINT binding sites, where the fluorescence signal peaks due to the transient binding of a fluorescent labeled DNA strand to its corresponding binding site within the nanoruler. Figure A1.13 has been added to Supplementary Material as Supplementary Figure 44. Q1.14. Figures S21 and S29 need better presentation. It is difficult to tell one plot from the other. In Fig. S21, if need be, authors may want to limit the number of frames to 25, but better visibility of each plot is quite important. Potentially a smaller but more important subset of SNR values may be used. A1.14. For Figure S21: To increase its clarity we have restricted the number of frames to 25 (see Figure  A1.14.1). In addition, a smaller subset of SNR is now being considered, this allows the reader to compare the effect of SNR in RSP and RSE more straight forward. Figure A1. 14 Figure S36). Note that, contrary to the theory that has been established in Supplementary Note 5.1.1, the FWHM of PSF to process these images is unusually high. This might be due to the size of the objects being larger than the PSF of the microscope itself (pixel-wise). Similarly, for the case of the comet-like structures analyzed in Supplementary Figure S37 (known to be~25 nm in diameter [43]), given that the pixel size of those images is very small and the information associated with the diffraction of the objects is spread across multiple pixels, a large FWHM of PSF must be used (see Figure A1. 15.2). This phenomenon is subject to further examination in the future.  Additionally, a new subsection (10.4) has been added to the Supplementary Material in order to support our discussion over MSSR providing image quality enhancement for SPT applications. Moreover, this new supplementary note serves as an additional example of MSSR performance over a diverse range of biological samples. For this, live porcine kidney (LLC-PK1) cells stably expressing plus-end-tracking protein EB3 fused to mEmerald were imaged to investigate the growth dynamics of microtubules at the tip. Figure A1.15.2 below (also Supplementary Figure S37) shows a comparison between a diffraction-limited scene of live LLC-PK1 cells displaying microtubule dynamics and its sf-MSSR super-resolved counterpart. Q1. 16. In all, I appreciate the work presented in this article. However, the authors need to address my major concern on super-resolution claims. While super-resolution is attractive, this work holds value even without super-resolution claim on sf-MSSR and authors have done a good job in demonstrating the value. So, I believe that this manuscript is a significant contribution even without super-resolution claim and strongly encourage the authors to be technically correct on this matter. If this concern is addressed, and possible improvements on the minor comments are incorporated, I will be happy to recommend this manuscript for publication. A1.16. We thank reviewer #1 for his/her insightful comments and valuable suggestions. As part of this review process, we have provided new evidence that, both qualitatively and quantitatively, proves the capability of MSSR of pushing the resolution limit even further down to~120 nm from the processing of a single fluorescence image (demonstrated with both synthetic and experimental data). Even though we understand the concerns of reviewer #1, regarding whether super-resolution is in fact achieved through the processing of one image or not. The main message we want to provide to the reader is that MSSR (whether sf-or t-, of any order) extends/enhances the resolution attained with any tested application in optical fluorescence microscopy. We believe that the latter claim will have deep impact in the optical microscopy field, because it expands the global access to fluorescence nanoscopy as it can be potentially applied to any application of fluorescence microscopy, regardless of its experimental or instrumental complexity. We are excited to see further applications of MSSR processing in imaging context not addressed in this manuscript.

Response to Reviewer #2
Remarks to the Author: This work presents MSSR, a principle and an algorithm for sharpening a digital image generated from diffraction of point sources. Given that the diffraction can be modeled with a Gaussian spread (PSF G ), the pixel intensities I(p) is a superimposition of spreads from emitters. The goal is to identify the positions of the emitters, or at least reduce the spread. MS stands for mean shift, or sometimes called the "blurring process". The idea is that by blurring, one can sharpen an image. The phrase "sharpening by blur" is a procedure in Photoshop, at least since 2017.
Q2.1. The idea is that by selecting a "spatial bandwidth", h s , (and on top of that, a "range bandwidth", h r ), one can identify a set of intensities in the color space, compute their mean, and then perform a mean shift, or update each pixel's intensity to the mean. If the pixel is not too far from the position of the emitter, the mean shift with the same Gaussian kernel may be in the same direction of the gradient in the intensity space, and the intensity may increase. Otherwise, the mean may go in the opposite direction and the pixel may lose its intensity. It would be nice and also important to relate the transformation of the point spread function (PSF G ), essentially the sigma parameter, to the mean-shift parameter h r (or maybe h s ) in this work. This should be easy but is not done here.
A2.1. We appreciate this suggestion, it leads us to rethink the seminal conceptualization of theory which supports the MSSR principle and algorithms. We redefine the definition of h s . Before, its value was set as the distance to the first minimum on Figure S6 In Fourier space, the objective lens of a microscope act as a finite aperture, the the optical transfer function of a traditional microscope can conveniently be represented by a circle, the diameter of which can be linked directly to the maximum observable spatial frequency K 0 , where K 0 ≈ 1/PSF FWHM , reviewed in [42]. The PSF can be simulated as an Airy pattern according to the features of the optical system: wavelength, numerical aperture, and the refractive index of the medium. It can also be measured experimentally from imagining diffraction limited fiducial markers. The following figure shows the relationship between the h s and FWHM in a Bessel distribution. Figure   We deeply thank reviewer #2 for this suggestion. The change in the definition h s , had a profound impact on the overall resolution enhancement provided by MSSR. Hence, any MSSR reconstruction provided in Figures 1-9 of the main manuscript have been recomputed using the modified h s domain. The same rationale was applied to figures in both the Supplementary Material and User Manual. Note that, in all cases there is a noticeable increase in the resolution (or contrast) attained. The overall conclusions of the main manuscript remained without change.
Q2.3 (S19) defines the kernel density estimate at each pixel position. It may contain a typo: By V do you mean B? A2.3. Yes, a mistake was made here. We have now changed the formula S19 from Q2.4 Since the positions of pixels are the grid, the k s part of (S18), and the g s part of (S23) are constants and the two-component profiles k = k s . k r and g = k' s .k' r are unnecessary and indeed when you reach (S24), only intensity is left. g n in (S25-27) will be the same with or without k s or k' s . In other words, the only component in [x, y] that moves is y, and no mean shift is done on x.
A2. 4 It is true that the pixels are positioned in a grid, but the values of k s and g s are not constant in formulas (S18) and (S23), respectively. This is because k s depends on the normalized difference related to h s between the central pixel p 0 and the member p n of the neighborhood . * ( 0 ) Figure A2.4a shows the normalized difference related to h s in a neighborhood of radius equal to 5 pixels. Note that as you move away from the central pixel, the relative distance increases until you reach the maximum value equal to 1. If a Gaussian kernel is used, its values decrease as you move away from the center of the neighborhood, this fact is shown in Figure A2.4b. Figure A2.4 has been added to Supplementary Material as Supplementary Figure S1.
We wish to clarify that the uniform kernel is the only one that provides no variation in its values, which are always constant or equal to 1. Any other type of kernel, such as Gaussian or Epanechnikov, presents value variations along the neighborhood. A2.5. Answer A2.4 shows that there is a change in the values of along the neighborhood. Therefore, it makes sense to shift the values of along the neighborhood, which this leads to being greater than ∆ zero. For this reason, we believe that Supplementary Note 4 and its theorem, which relies on , is well ∆ justified.

New changes
Main manuscript -New figures: • Figure 3: sf-MSSR n extends spatial resolution in confocal microscopy, • Figure 4: sf-MSSR n enhances the resolution and contrast of Airy scan and SIM reconstructions. • Figure 6: sf-MSSR n enhances spatial resolution in STED microscopy. Note that when adding figures, the actual number of these has been shifted compared to the previous version.
-Most of the figures have been updated with the mpl-infierno LUT using FIJI/ImageJ. -Some texts have been rewritten in response to the reviewers and improve the overall quality of the article.
-Changes in the document have been highlighted in red. -An option to select the type of interpolation has been added.

Manual
-An option related to perform intensity normalization on sequence of images has been added.
-The selection of batch analysis can now process several images in a given folder.
-Codes for Matlab, R, and Python have been updated according to the new changes on the MSSR theory.